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ABSTRACT 

A model for evaluating the counseling interview is 
presented. It is felt that this model is both comprehensive and 
systematic in that it relates the kinds of parameters that appear in 
the counseling literature to the processes and outcomes of 
counseling. The three process dimensions that are presented include 
the client, counselor, and milieu factors. Several research articles 
which illustrate specific variables in terms of the three process 
dimensions and their interactions are cited. A categorization of 
counseling outcome dimensions is provided. The outcome dimensions 
include affective and cognitive reorganization and behavior chemge. 
In order to relate counseling processes to counseling outcomes, a 
rationale which applies multiple linear regression techniques using 
process variables to account for variance in specific, counseling 
outcomes is described. A checklist of process and outcome variables 
is provided to aid the evaluator in selecting parameters to include 
in the evaluation of a specified counseling program. (Author) 
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ABSTRACT 

EVALUATING THE COUNSELING INTERVIEW— A MODEL 

Richard A. Gilberts 
Marcella Sherman 

CENTER FOR PLANNING AND EVALUATION 
Santa Clara County Office of Education 
1110 North Tenth Street 
San Jose, California 95112 

A model for evaluating the counseling interview is presented which is 
comprehensive and systematic as it relates the kinds of parameters that 
appear in the counseling literature to the processes and outcomes of counseling. 
Three process dimensions are presented and include the factors of: 1) Client, 

2) Counselor, and 3) the Milieu. Several research articles are cited which 
illustrate specific variables in terms of the three process dimensions and 
their interactions. Secondly, a categorization of outcome dimensions of 
counseling is provided which include: 1) Affective re-organization, 

2) Cognitive reorganization, and 3) Behavior chauge. 

So that counseling processes may be related to counseling outcomes, a 
rationale is described which applies multiple linear regression techniques 
using process variables to account for variance in specific outcomes of 
counseling. A checklist of process and outcome variables is provided to 
aid the evaluator in the selection of parameters to include in the evaluation 

- s*.. 

of a specified counseling program. 




I' 

X. 



1 



FILMED FROM BEST AVAILABLE COPY 



EVALUATING THE COUNSELING INTERVIEW- -A MODEL 

Richard A. Gilberts 
Marcella Sherman 

CENTER FOR PLANNING AND EVALUATION 
Santa Clara County Office of Education 
1110 North Tenth Street 
San Jose, California 95112 



Introduction 

Counselors and psychotherapists believe deeply in the efficacy of their 
professional activities. They believe that they are capable of assisting their 
clients in many ways: depressed are cheered; the anxious are taught to cope; 

the withdrawn overcome the traumatic; and the anti-social are relieved of some 
miserable habits. Most counselors are prepared to say that they help people to 
deal effectively with their problems through applying special techniques acquired 
through training and experience. Yet Eysenck (1952) said that time heals just 
as well as psychotherapy. 

In the lS60's, counseling researchers began to document the claims of 
counselors and psychotherapists. Rogers and his students (Truax and Carkhuff, 
1963) showed that counseling had an impact on clients, often influencing client 
growth. Krumboltz and his colleagues at Stanford produced evidence that 
counseling had observable effects on the behavior of students beyond the confines 
of the counseling interview (Thoreson and Krumboltz, 1967). 

More recently, with the growth of the accountability movement in government, 
health services, and education, the old questions are being raised again and 
additional questions are being asked: how can client gains be measured? What 
is the relationship between client benefits and the costs of counseling services. 

Public school personnel particularly have been raising questions about the 
effectiveness and perhaps the quality of counseling practices in their domain. 
School teachers and principals have not been in full agreement that schools are 
the proper place four counseling activities. Some school officials as well as 
their constituents question whether taxpayers should be expected to foot the 
bill for services beyond instruction in the basic skills. 

In order that the profession of counseling psychology can maintain a per- 
spective of its own position and communicate to various publics the benefits 
that accrue from its services, it is necessary that some procedures for providing 
information to decision-makers be formalized so that counseling programs may 
be validly observed. It is the purpose of this paper to: 1) outline a model 

for 3 comprehensive evaluation of the counseling interview; and 2) to suggest 
a method of evaluating the psychological interview. 




Assumptions of the Model 

Before the model is presented, it will be helpful to illuminate the assump- 
tions that underlie its structure. In a general sense, it is assumed that 
counseling activities are intensely personal and complex. While some single 
factors may directly affect some clients, it is more often the case in the 
counseling process that complex interaction effects are contributing to changes 
in the behavior of clients. For this reason, the model to be described leans 
heavily on the operation of interaction effects both in the process factors of 
counseling and in the outcome factors. 

Secondly, the model assumes that evaluation of counseling ought to be 
descriptive rather than predictive. Because of measurement errors, recording 
errors, sampling errors (Horst, 1966) and more recently, the unreliability of 
gain scores (Stake, 1S71), it is advisable that the evaluation of a specified 
psychological interview be limited to the group under observation. 

Thirdly, valid and reliable observations of the counseling process are 
not confined to quantitative instrumentation. Modifications of client behavior 
can be described, in many cases persuasively, using narrative descriptions of 
the significant people and their psychological environments. It is easy to 
foresee the selection of parameters and the effective use of persuasion in 
the application of the model to be described to a particular counseling program. 

Finally, it is assumed that the evaluative process and product will reflect 
the biases of the evaluators. Evaluators of the psychological interview are 
likely to be involved in some way in the therapeutic enterprise. Whatever 
variables are selected for attention, whoever the client might be, and however 
the interview is monitored, it can legitimately be expected that the evaluator 
bias will be embedded in the findings and the interpretation of data. However, 
one should not be unduly alarmed at this prospect since the outcome of the 
evaluation could include a sensitive appreciation of the essential changes on 
the part of the clients, sophisticated methodology, and insightful strategies 
for improving the therapeutic world. 



The Evaluation Model 



The basic structure of the model for evaluating the counseling interview 
is the familiar independent and dependent variables that constitute the multiple 
linear regression paradigm. This kind of schema provides for comprehensiveness 
since any number of variables significant to therapeutic processes may be 
considered. Further, the model is methodologically explicit as it lends itself 
to the application of multiple regression theory. 

Figure 1 illustrates the basic configuration of the model. The cube on the 
left displays the three principal dimensions or ingredients of therapeutic 
activity. These factors are labeled process variables. Of course, the counselor 
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the client, and the milieu*, or the social conditions in which the counseling 
interview occurs represent significantly broad factors in which more specific 
attributes of the therapeutic condition may be evaluated. While the vertical 
lines on the front surface portray specific client characteristics, the horizontal 
lines of the cube indicate counselor variables. The third dimension of the 
social milieu is shown to be composed of more specific social conditions (i.e., 
group and individual settings) and associated methods (i.e., desensitization 
and verbal conditioning). All of the aforementioned attributes constitute the 
processes of counseling activity. 

Appearing on the right side of the model are the outcomes of counseling. 

The three principal components of the outcome side of the paradigm are labeled 
as follows: 1) Cognitive reorganization, 2) Affective reorganization, and 

3) Behavioral change, a framework suggested by Stefflre (1970). Traditional 
ende avo rs, in counseling research and evaluation have largely neglected the 
outcome phase of counseling. 

Implicit in the dimensions of both process and outcome are the interactions 
between factors. Interactions between the various factors rather than the 
effects of only discrete factors ought to be considered more frequently in 
psycho-therapeutic evaluations. For example, we should pay greater attention 
to counselor /client interaction as both dimensions merge to influence some 
specified outcome. Other important interaction effects are the counselor/ 
method and the client/method interactions. It is conceivable, for example, that 
the kind of person the counselor is may be dependent on the method he employs. 
Specific variables of each of these processes and outcome dimensions are provided 
in the subsequent section. 



Specific Process Factors 

The model has been described in terms of the larger dimensions of the 
counseling interview. The reader may be interested in some of the specific 
characteristics of the three-dimensional figures on both sides of the paradigm. . 
The counseling literature is rich in information regarding the pieces of the 
puzzle. An attempt is made here to fit the pieces into the larger framework 
of the model. 

The studies in this article are only suggestive of a larger population 
of research articles which bare upon the evaluation of the psychological 
interview. Literature reviews by Cartwright (1968), Thoreson (1969), 

Patterson (1966), Hosford and Briskin (1968), Stefflre (1970), and Island (1968) 
provided the sources for identifying attributies of the interview in the 
psychological literature. The reader is referred to the foregoing reviews 
for a more integrated view of counseling research. 




* Some would label this third dimension "counseling method" rather than milieu. 
These waiters perceive the social milieu to sumsume the method-. 
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CHent Factors (I) 

Predispositions of clients seem to be highly regarded topics for inquiry 
in present research reviews. Variables such as social class, sex preferences 
for counselors, readiness for counseling, and client expectations, are shown 
in Figure 2, together with instruments used to observe the variables, results 
of specific research, and respective authors. 

Therapeutic programs may be established directed at specific target groups 
such as lower class populations. Success of the program may be predicated on 

unorthodox methods of therapy such as suggested by Gould (1967) which include 

being more informed, more flexible about lengths of sessions, meeting patients 
in places other than the office (i.e., a bar, the park), being more physically 
active and doing more standing and moving around. Observation of the clients' 
SES is important and may contribute to the ultimate outcomes of the program. 

Other variables related to the client factor reported in the reviews were 
sex preferences for counselors on the part of students, readiness for counseling, 
clier.it expectations for the interview and birth order, and whether the client 
volunteered for counseling or was referred. Some interesting instrumentation 
to assess these propensities included a Counseling Readiness Scale, the Truax 
Depth of Interpersonal Exploration and the Kyers-Briggs Type Indicator. The 
Counseling Readiness Scale was developed on clients with personal problems to 
predict counselee continuance in counseling (Heilbaun and Sullivan, 1962) . 

The Truax Scales (Truax, 1962) attempts to measure depth of interpersonal 
explorations in the interview; and the Heyers-Briggs type indicator was 
developed to assess a kind of thinking/feeling dimension in clients (Mendelsohn 
and Geller, 1963). 



Counselor Factors (II ) 

Characteristics of counselors in the research literature is prolific.' 

These qualities seem to cluster around theoretical orientation, personality 
traits, and experience. Figure 3 portrays some studies that illustrate the 
kinds of observations that have been made, the instruments used, the results 
of the research, and the respective authors. Each of these counselor 
idiosyncrasies may be determinants of counseling outcomes and are to be placed 
in the columns of the front surface of the model in Figure 1. 

v 

The question of theoretical orientation of counselors effecting the 
behavior of clients was raised in the Fiedler studies (1950a and 1950b). He 
contended that the theoretical views were not important in influencing counselor* 
behavior in interviews. Other investigators have since questioned his findings 
(Sundiand and Barker, 1962; McNair and Lorr, 1964) to extent that Stefflre (1970) 
contended that Fiedler's conclusion does not seem as tenable as it once did. 
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THE CLIENT FACTOR IN 


EVALUATION 






Observation 


Instruments 


Resul ts ' 


Authors 




1. Social class 


Therapy tapes 


Lower classes talk 
more about phy- 
sical ailments 


White 

(1966) 




2. Sex preference for 
counselors by 
students 


Questionnaire 


Same sex pref- 
erence for clients 


Fuller 

(1964) 


> 


3. Client preferences 


Rokeach Dogmatism • 
scale 


No tendencies for 
pairs to select like 
co's. 


Steffler & 
Leafgreen 
(1963) 




4. Counseling 
Readiness 


Counseling Readi- 
ness scale (CRS) 


Clients with inter- 
view longevity 
are less self- 
accepting, less 
able to assume 
responsibility 


Heilbrun & 
Sullivan 
(1962) 




5. Expectations 


Questionnaire 


Clients' expecta- 
tions change as a 
result of counsel- 
ing toward wanting t 
more active 
participation 


Pohlman 

(1961) 




6. Birth order 


Interview 


First-borns spoke 
more frequently, 
asked more 
questions and 
talked more when 
anxious 


Ei8enman 

(1966) 




7. Volunteers 


Myers -Briggs Type 
Indicator 


College male vol. 
concerned more with 
perception and in- 
tuiton than judg- 
ment and sensation 


Mendelsohn 

(1962) 




8. Volunteers 

i 


Mooney 

Problem Checklist 


Volunteers differ 
from referrals: 
more females, more 
problems , higher 
GPA, problems with 
people 


Esper 

(1965) 


|er|c 


9. Client explora- 
tion 


Truax* s Depth of 
Interpersonal 
Exploration Scale 

■v* 7 


May be applied to 
tape recordings or 
transcripts 


Truax 

(1963) 
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THE COUNSELOR FACTOR IN EVALUATION 



Observation 


Instruments 


Results 


Authors 


1., Theoretical 
Orientation 




Not a determanent of 
therapist behavior 


Fiedler 

(1950) 


2. Counselor 

Orientation 


Therapist Orienta- 
tion Scale 
(analytic- 
experimental) 


A general factor: 
analytic scale and 
experimental scale 
account for thera- 
pist differences 


Sundland 

Barker 

(1962) 


3. Counselor Orien- 
tation 


MD scale derived 
from TOS 


3 factors: 
psychoanalytic 
impersonal vs. 
personal, direc- 
tive-active 


*McNarv < 
Lorr 
(1964) 


4. Experience 


Gulford-Zimmerman 


No diff. between ex- 
perienced & inexper- 
ienced counselors 
subrole 
behavior 


Campbell 

(1362) 


5. Experience 


Discharge rate, 
psychological tests 


Young, untrained col- 
lege girls effected 
improvement of male 
schizophrenics sig- 
nificantly better 
than trained 
psychiatrists 


Poser 

(1966) 


6. Experience 


Drop out rate from 
interviews 


More for inexperienced Baum 
co. (maturity is (1966) 

crucial) 


7. Openness 


Rorschachs 
scored for re- 
pressive styles 


Counseling effective- 
ness correlated with 
openness rather than 
training 


Allen 

(1967) 


8. Interests 


Strong interest 
inventory 


Therapists were high 
on lawyer & CPA 

* 


Betz 

(1961) 
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Figure 3 (Cont* d. ) 

THE COUNSELOR FACTOR IN EVALUATION 



Observation 


Instruments 


Results 


Authors 


9. Counselor Person- 
ality 


MMPI, EPPS, SVID 


Nat. sample of rehab 
co. students (high K, 
KC, Ma), hi inter- 
ception, psychol- 
ogist-social wel- 
fare patterns 


Patterson 

(1962) 


10. Counselor Person- 
ality 


The ways of life... 
self -description 
checklist 


NDEA Inst. Ss 
emerged as con- 
trolled confor- 
mists and repressed 


Mahan 

(1964) 


11. Counselor Person- 
ality 


Rolce a c h 1 s Do sta- 
tion scale 


Coached client rat- 
ings showed better 
counselors to be 
less dognatic 


Russo, Kelz 
& Hudson 
(1964) 
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The kinds of issues related to personality attributes of counselors as 
they effect their clients in interviews seem to capitalize on many of the well 
known personality tests (i.e., MMPI, EPPS, SVIB) used to monitor changes in 
personality accompanying a training program. The Poser (1966) study showed the 
effects of experience on patient outcomes. Poser concluded that training may 
not be necessary for the promotion of therapeutic behavior after showing that 
young untrained college girls were superior to experienced psychiatrists and 
psychiatric social workers in influencing 343 chronic male schizophrienic 
patients. Cartwright (1970) was hard pressed to understand what relevance these 
studies have for psychotherapy training as it is usually understood. 

In the evaluation model any number of counselor qualities, such as the 
ones suggested, may be selected to assess their relationships to changes in 
clients depending on the proclivities of the program or the theoretical 
embracings of the research evaluator. 

i 

Milieu Factor (III) 

Two themes dominate the milieu dimension of the counseling processes. The 
first theme deals with the number of clients participating in the interview: 
individual and group counseling interviews being the primary constituents. 

The second is a counseling method theme. Included are the methods of- 
sensory deprivation, use of silence, vicarious therapy, desensitization, 
video-tape playback to clients, and the psychological milieu. Figure 4 shows 
the pertinent information relative to the observations, the instrumentation, 
the results of research, and authors of research of the milieu dimension. 

One of the more interesting methods was the Adams' et al. , studies (1966) 
which applied sensory deprivation to psychiatric inpatients to facilitate self- 
understanding and insight. The reported results showed decreased defensiveness 
and increased ego-strength and an increased MA score using the IftiPI and the 
Interpersonal Checklist before and after treatment. If one of the stated 
objectives of a counseling program is to facilitate self-understanding and 
insight, Adams' paper could provide a useful strategy in the overall evaluation 
design. If, on the other hand, the intent is to generate modification in 
client behavior outside the interview, the transfer effects detected by 
Krumboltz and Thoreson (1967) would be useful to incorporate as a milieu factor 
in the design of the program. 



Interaction Factor 



Perhaps the process interaction most frequently studied is the one between 
the client and the counselor. Variables such as SES, theoretical orientations, 
levels of functioning, counselor style, verbal behaviors, counselor acceptance, 
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Figure 4 

THE MILIEU (METHOD FACTOR IN EVALUATION 
Observation Instruments Results Authors 



1. Interview type 



2. Transfer effect 



3. Sensory 

deprivation 



4. Psychological 
Milieu 



5. Silence 



6. Vicarious 

Therapy pre- 
training 



Interview 



Observation 



MMPI - Inter- 
personal 
checklist 



Accurate Empathy 
Scale, Uncondit- 
ional Positive 
Regard Scale, 
Therapist 
Genuineness or 
Sel f- Congruence 
Scale 



Low anxious Ss re- Kaplan 
sponded signifi- (1966) 

cantly more to the 
pro b Lem ‘ focused 
interview while low 
anxious Ss responded 
more to the free 
association inter- 
view 

Reinforcing infor- 
mation seeking R ? s 
outside of 
therapy 

Deceased defensive- Adams, 

ness, increased Robertspn 

ego strength and & Cooper 

increased Ma score (19'G6) 

after sensory 
deprivation 

The scales can be Truax 

applied to tape re- (1962) 

cordings of 
counseling inter- 
views 



Thoreson & 
K'" i-mbol tz 
(1967) 



Process Scale of Silence was assoc- Cook 

Truax iated with success. (1964) 

Silence may be an 
index of empathy, 
congruence and 
positive regard 

Client's listen to Has facilitating Truax 

tapes of "good" effects in counsel- (1963) 

client behavior ing 



ERIC 



11 



11 



figure 4 (Cont'd.) 

THE MILIEU (METHOD) FACTOR IN EVALUATION 



Observation 



Instruments 



Results Authors 



7. Videotape play - Use 1st interview No results yet 
back recording in 2nd 

interview, etc.^ for 
succeeding inter- 
vievjs, client asked 
to take role of co. 



Schmedling 

(1962) 



8. Group Counseling Sematic Differential Boys who received 

Wohler's Self Do- modeled role play- 

scri.pti.on Inventory E rou P therapy 

changed most 



Sarason 

(1968) 



. 9. Desensitization Behavioral Accor 

dance Test 



Standardized desen- Fishman & 

sitization schedule Newas 
reduced tear was (1971) 

found effective in 
group treatments 
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client expression of feeling, rapport, and transparency arc illustrative of the 
qualities observed in both the client and the counselor as they effect ch^nj;,o ^ 
in the client. This kind of data is sham in Figure 5. For instance, Welkowitz 
and Ortmeyer (1967) investigated the value system similarity between patient 
and therapist. These authors found that showing similar values correlated 
best with improvement in clients. 

The convergence hypothesis supported by Pepinsky and Karst (196*':), which 
•'states - that- clients • improve in therapy because they change to become more 
like their counselor, exemplifies this important interaction. While the 
convergence hypothesis is not as tenable as it once was (Stefflre 9' ), t e 
relative importance- of the c lient/counselor interaction is shown by this kind 
of conceptualization. 

The virtual’ absense of counselor /method interaction as they affect 
outcomes in counseling is conspicuous. It may be worthwhile knowing in a 
particular evaluation that counselors with personality propensities wort est 
with certain counseling techniques and not so well with others, or that some 
therapist work better in given social conditions. At any rate, some attention 
needs to be invested in the counsclor/milieu interaction. 

A study illustrating the interaction of the client/milicu interaction 
investigated the effects of interview; structure or. different levels of 
anxiety in subjects (Kaplan. 1966). Tne results shevesd that lgi y anxious 
subjects responded significantly more to problem-focused interviews while 
low anxiety level subjects responded more to free association interviews. 

Other characteristics of client milieu interaction which produces changes 
in clients are needed. It should be noted that the three-way interaction of 
counselor/clieVit/milieu factors were also neglected in the counseling research 

literature. 



Specific Outcome Factors 

The dimensions of outcomes have already been characterized as consisting 
of: 1) Cognitive reorganization, 2) Affective reorganization, and 3) Behaviorial 

modification. Since different approaches to measuring client change are 
acceptable to various professionals and laymen alike, it makes considers e 
sense in model construction to provide as many alternatives as possible for 
considering change in clients having experienced therapy. 

One of the frequent complaints of those reviewing the counseling 
literature is the inadequacy of criteria available for measuring c anges 
resulting from counseling. Cere, .right (1968) referred to the 

problem as an unsolved centre! issue in the field. The purpose of he evalua- 
tion model is not to resolve the criterion problem, but rather , , . 

various approaches to measuring change in a new perspective. It is critical 
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Figure 5 



COUNSELOR/CLIENT INTERACTION IN EVALUATION 



' Observation Instruments 



Results Authors 



SF.S (CI )- Experience 
(C) 

SES in client & 
counselor 


Dropout rate from 
therapy 

Q sort based on 
EPFS items index 


More lover class cl 
dropouts v;ith inex- 
perienced co’s. 

Mora educated clients 
were helped by 
higher trained pro- 
fessionals (i.e. 
psychiatrist) 


Baum, et al. , 
(1965) 

Keith- 

Spiegel 

(1967) 


Counselor /client 
value similarity 


Morris Ways to Live. 
Test ai;d SVII 


Sharing similar 
values correlated 
best with improve- 
ment in clients 


Welkovritz, 
et al., 
(1967) 


Theoretical 

orientation 


Rogers & Rablen 
process scale 
Bales Int. Proc. 
Analysis 


Similar clients (or 
the kir.d of client) 
moved similarly in 
therapy regardless 
of differences in 
technique 


Cartwight 

(1966) 


Co: empathy, re- 
spect, & regard 
Cl; functioning 
level 


Truax scales 


Hi functioning 
clients are more 
independent of co. 
behavior than low 

function 'ft persons 


Holder 

(1967) 


Sex 




Female clients ex- 
press more feeling 
than male regardless 
of counselor sex 


Fuller 

(1964) 


Counselor style 
(leading & re- 
flective) and 
induced client 
set 




Co's, not consistent 
in their co. style 


Rottschaffer 

(1962) 


Client co. simil- 
arity (assume 
ability to 
comm. ) 


Myers-Briggs Type 
Indicator 


Cl-co, similarity 
correlate with no. 
of counseling 
sessions 


Mendelsohn 

(1963) 
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Figure 5 (Cont'd.) 



COUNSELOR/CLIENT INTERACTION IN EVALUATION 



Observation Instruments Results Authors 



9 . Therapist/client 
transparency 


Truas* s genuiness 
scale 


Therapist transpar- 
ency correlated 
with client trans- 
parency 


Truax 

(1965) 


10. Co./cl values 
similarity 


AVSV 


Duration and success 
related to Co.-Cl. 
similarity 


Cook 

(1966) 
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to people evaluating the processes and outcomes of counseling that the relation- 
ships among various sets of criterion measures be seen in perspective. 

In the model being presented, no hierarchial scheme is superimposed on the 
three o-tcomc dimensions. However, it is to be expected that professionals 
imbued with the centrality of consciousness or being in human affairs will tend 
to. apply criteria that measure the affective domain. Conversely, those who are 
convinced that human actions are the best indicators of change occuring in 
clients subsequent to therapy will likely select measures that fit the behavioral 
dimension of change. While the points of view tray be quite irreconcilable, 
nevertheless both measures may be admitted in an evaluation strategy as 
evidence of change. For a thoughtful analysis of differing foci on the measure- 
ment of change in counseling, see Ilosford and Briskin (1968). 



Cognitive Outcomes 



A change in the cognitive structure characterizes the criteria applied to 
this factor. Whether the change be gains in academic achievement, vocational 
choice, or increased information, the client will sec the world differently. 
Some of the criteria used to assess this dimension are grade point average and 
achievement tests. While Hill (1966) perceived GPA as a poor measure of 
cognitive change, it is likely that achievement measures will be used more 
frequently in future research and in evaluation programs as an outgrowth of 
the accountability movement and the associated public interest in improving 
basic skills. Variables measured and results of studies specific to cognitive 
changes are provided in Figure 6. 



Affective Outcomes 



The discriminating feature of this factor is that changes in the self- 
consciousness occur as a function of the therapeutic processes and the client 
is different because of new insights gained or psychological stance modified. 
One can anticipate affective domain criteria to include measures of self- 
actualization, openness to experience, self-esteem, self-exploration, and value 
systems. Personality assessment inventoried of a clinical kind are frequently 
found in these instances. Illustrative studies may be found in Figure 7. 

If clients can be affected to feel better about their life circumstances, 
if they can be assisted to be less fearful and anxious in their interpersonal 
relationships, or if they can be persuaded to be understanding of childhood 
experiences, then it may be anticipated that these internal conditions will 
manifest themselves in improved theoretical constructs end their related 
measurements. 
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Figure 6 

COGNITIVE REORGANIZATION CRITERIA IN COUNSELING EVALUATION 



*. Observation 



Instruments 



Results Authors 




1. GPA 



GPA 



2. Academic per- 
formance 



3. Reading 

achievement 



4. Achievement and 
career 



Cooperative Elg. 
Ach. Tests 
Occupational As 
piration Scale 
Voc. develop 
Inv. 



Counseled women coll. Ivey 
students improved GPA (1962) 

significantly 

Hi anxious students Spielberger, 

who volunteered Weitz & 

for group co. im- Denney 

proved significantly (1962) 

Sig. diff. between Dolan 

groups receiving (1964) 

indep. co. and 
matched non' 
counseled group 



Black adolescent Gilliland 

students: group (1968) 

counseling improved 
scores on achieve- 
ment tests 



5. Underachieve- GPA 
ment 



Improvement for coll, 
freshmen with 24 
group therapy 
sessions 



Dickenson & 
Truax 



(1966) 



6. Underachieve- 
ment 



7. Appropriate- 

ness of 

vocational 

objectives 

8. Information 

about tests 
& vocational 
choice 

9. Information 

seeking be- 
haviors 



GPA 



Projective story 
completion 



10th grade boys Benson & 

improved 18 weeks Blocher 

of group counseling (1967) 

Counseled Ss had more Gonyea 
appropriate voc. (1962) 

objectives after 
counseling 

Counseled group im- Wright 

proved more than (1963) 

non-counseled 
group 

Conditioning general- Ryan & 
ized to non-counsel- Krumbultz 
ing talk (1964) 
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Figure 7 

AFFECTIVE REORGANIZATION CRITERIA IN COUNSELING EVALUATION 



Observation Instruments Results Authors 



1. Self-actualiza- 
tion 



2. Openness to 
experience 



3. Anxiety 
reduction 



Personal Orientation 
Inventory 



Fitzgerald Scale 



IPAT anxiety 



Significant increase 
in self-actualiza- 
tion 

High scores predict 
original word as- 
sociation and 
ability to shift 
responses under 
different 
conditions 

Patients had high 
initial levels 
and E dropped 
more than C 



A. Self-Esteem Rorschach Self 

& Ideal Q sort 



5. Self-Exploration MMPI 



6. Moral Values 



Kelly's Role 
Construct 
Repertory Test 



No differences be- 
tween E-C groups 
after 20 sessions 
of therapy 

Counselor trans- 
parency was 
related to 
client self- 
exploration 

Clients moved 
toward the ideal 
of the therapist's 
values in the 
clients perspective 



Lieb & 
Snyder 
(1367) 

Fitzgerald 

(1966) 



Cattell 

(1966) 



Gay 1 in 
(1966) 



Mattarazo 

(1965) 



Landfield 6c 
Newas 
(1964) 
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Behavior Outcomes 



The most salient feature of the behavioral change dimension is the specificity 
of the observation. When the task of counseling is highly visible and the 
assessments can be made by counting the frequencies of response of the subjects, 
the appropriate criteria to apply are behavioral criteria. 

Measures of this kind are highly reliable. Illustrating these kinds of 
variables (see Figure 8) are delinquent behavior, extinction of phobias and 
undesirable habits, modification of verbal behavior and the like. Very often 
behavior modification techniques are used in the interview such as systematic 
desensitization and reinforcement schedules. 

It is very possible that a change in one dimension is dependent upon change 
in another. For instance, a change in behavior, such as the extinction of a 
phobia, may be accompanied by a shift in attitudes and cognitive structure. A 
school phobic who, subsequent to a treatment program, now attends school is 
likely to improve in his academic performance as well as to increase his liking 
for teachers and classmates. The interaction of outcome criteria presents some 
very real possibilities for evaluation. 



Methodology 

Implementing the counseling evaluation model in an evaluation program may 
be accomplished by first transforming selected process and outcome parameters 
into objectives and second, applying appropriate data analysis techniques to the 
obtained data. 



Objectives 

Mager (1962) has explained at length some of the ingredients of objectives 
for evaluators. More recently O’Hare and Lasser (1971) have presented a 
rationale for evaluating pupil personnel programs, and Wellman (1967) has 
provided a comprehensive listing of objectives together with criteria for 
assessing them in a taxonomy for guidance programs. 

In a discussion of evaluating a behavioral counseling program, O’Hare and 
Lasser (pp. 71-77) presented a case study describing how Greg, an eleven year- 
old fifth grader, was trained to extinguish his disturbing and inappropriate 
classroom behaviors. After a system of reinforcement was applied to Gre g and his 
classmates, the criteria of diminishing the outbursts to one per hour was 
sufficient for asserting that the treatment for Greg was successful. Often 
in the evaluation of programs, a successful evaluation is predicated on whether 
the objectives stated prior to the implementation of the treatment w^re accomplished 
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Figure 8 



BEHAVIOR CHANGE CRITERIA IK COUNSELING EVALUATION 



Observation Instruments Results Authors 



1. Delinquent TAT 

Behavior 



2. Phobia Cornell Index 

Extinction 



3. Insomnia Case study 

Reduction 



A. Stealing Case study 

Extinction 



5. Verbal Behavior Projective Story 

Completion Task 



6. School Phobia 



Institutionalized 
offenders obtained 
significantly 
lower anti-social 
scores 

More rapid improve- 
ment with reduced 
time using de- 
sensitization 



Treated successfully 
in 14 sessions with 
desensitization 

Ten year old boy 
develops maturity 
and age appropriate 
behavior 

Decision and deli- 
beration R' s were 
reinforced and per- 
sisted in non- 
counseling tasks 

Desensitization in 
the school environ- 
ment proved 
successful 



Johnson 

(1961) 



Gelder , 
Macks, & 
Wolff 
(1967) 

Greer & 
Katkin 
(1966) 

We tz e 1 
(1966) 



Ryan & 
Krumbultz 



(1964) 



Garvey & 
Hegrenes 
(1966) 
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In light of the counseling evaluation model already presented, imposing 
objectives on Greg's treatment commensurate with the cognitive or affective 
dimensions of counseling outcomes could have enhanced the evaluation of this 
event. Greg most likely had a change of attitude as a result of his treatment 
or possibly he saw his classmates and teachers from a different perspective. 

There may have been an associated increase in his self-esteem or he may have 
become anxious in his new school environment now that his disruptive behavior 
was diminished. It is also possible that his academic performance was improved. 
More sense could be made of Greg's treatment program if other kinds of information 
were available. Did ending the disruptive behavior improve the quality of his 
life? This question and others ought to be asked, in a valid evaluation of the 
treatment . 



Data Analysis 



The counseling research literature speaks directly to the problem of dealing 
effectively with problems of research design. Stef five (1S70) and Thoreson 
(1968) exposed the need for gaining controls over variables by comparing 
competing counseling treatments and the use of single criterion measures. 

Practical problems facing researchers and evaluators such as proper randomization 
of subjects, controlling treatments, and protection against contamination combine 
to mitigate against reliable and valid investigation. However, all is not lost. 

Walberg (1971) described how multivariate multiple regression techniques may 
be used to sort out incremental, multivariate, interactive, and non-linear effects 
inherent in much of the research in counseling. Analysis of variance designs, 
Walberg (p. 72) contends, which compare differences between treatment (i.e., 
counseling programs) variables or response classes (i.e., personality traits) 
may be arranged to accommodate a multiple regression analysis equation. 

The advantage of this approach is that the amount of variance in outcome 
variables can be assigned separately to treatment programs and response classes. 

By attending to the percentage of variance accounted for in the outcome, measures 
by process measures, the effects of counseling treatments, personality 
characteristics of clients and counselors, and milieu factors can all be 
accounted for separately in regression analysis techniques. Quasi experimental 
programs (subjects are assigned to treatments by convenience) as in the case in 
many evaluation programs, are particularly amenable to regression analysis where 
assumptions of experimental randomization is not critical. The test of whether 
the input variables are contributing significantly to variance in the outcome 
measures is made by computing F ratios through testing the increments in the 
squared multiple correlation, an estimate of accountable variance. 

Perhaps the most salient feature of multiple regression analysis is that 
many input and many outcome measures of counseling effectiveness may be applied 
in a given evaluation program. A consideration of many ingredients of the 
counseling process can be viewed in the light of a substantial number of outcomes 
of the program. 
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The convenience of using multiple regression should not be overlooked. Most 
university ar.d government computer facilities have readily available computer 
programs and other software required for multiple regression analysis procedures. 
One popular version is the Stepwise Regression BMD02R of the Biomedical Computer 
Program (1965) package available for users at most of the larger computation 
centers. This program computes statistics essential for ranking the ingredient 
variables as they contribute to variance in the outcomes, beginning with the 
most powerful variable and adding to that the next most powerful in terms of 
variance accounted for in the outcome until some criteria is met. However, other 
multiple regression rationals are also appropriate. 

Horst and HacEwcn (1960) have described a method of multiple linear 
regression where an elimination procedure is used in contrast to the accretion 
procedure available in the BMU02R program. The elimination procedure considers 
the entire pool of input measures and systematically eliminates those measures 
in order of least predictive value for the specified criterion.. A computer 
program in Fortran IV was developed by Gilberts (1968) which is based on the 
elimination technique. 



A Counseling Objective Checklist 



To facilitate the organization of selecting objectives, a checklist is 
provided in Figure 9 that illustrates the kinds of parameters significant to 
counseling processes and outcomes. Host evaluations of counseling programs will 
plan to consider only special kinds of process and outcome variables. The 
kinds selected will depend upon the long-range commitments of the program, the 
nature of the clients, available facilities, and the theoretical inclinations 
of the evaluators or therapists. 
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